Evaluation  Study  of  Mesoscale  Models  MM5  and  BFM  Over 
the  Model  Domains  of  Utah  Using  Surface  Meteorological 

Data  by  Mesowest 


by  Teizi  Henmi 


Approved  for  public  release;  distribution  is  unlimited. 


20030312  112 


NOTICES 


Disclaimers 

The  findings  in  this  report  are  not  to  be  construed  as  an  official  Department  of  the  Army  position  unless 
so  designated  by  other  authorized  documents. 

Citation  of  manufacturer’s  or  trade  names  does  not  constitute  an  official  endorsement  or  approval  of  the 
use  thereof. 

Destroy  this  report  when  it  is  no  longer  needed.  Do  not  return  it  to  the  originator. 


Army  Research  Laboratory 

White  Sands  Missile  Range,  NM  88002-5513 


ARL-TR-2928  _ March  2003 


Evaluation  Study  of  Mesoscale  Models  MM5  and  BFM  Over 
the  Model  Domains  of  Utah  Using  Surface  Meteorological 

Data  by  Mesowest 


Teizi  Henmi 

Computational  and  Information  Sciences  Directorate,  ARL 


Approved  for  public  release;  distribution  is  unlimited. 


Contents 


Executive  Summary  1 

1.  Introduction  2 

2.  Description  of  the  Models  and  Model  Domains  4 

2.1  MM5 . 4 6 

2.2  BFM . 7 

3.  Meteorological  Data  9 

4.  Forecast  and  Evaluation  Methods  11 

4.1  Forecast  Method . . . . . 1 1 

4.1.1  MM5 . 11 

4.1.2  BFM . 11 

4.2  Evaluation  Method . 12 

4.2. 1  Data  Comparison  Method . . 12 

4.2.2  Statistical  Parameters . 12 

5.  Results  I4 

5 . 1  Statistical  Parameters  of  Daily  Data . 1 4 

5 .2  MM5  Forecast  and  Observation  Data  Comparison . 15 

5.3  Statistical  Parameters  of  Entire  Forecasting  Periods . 21 

5.4  Time  Series  of  the  Statistical  Parameters . 22 

6.  Conclusions  27 

References  29 

Acronyms  7^ 

Report  Documentation  Page  80 


111 


Appendix  A.  Statistical  Diagrams  Displaying  the  Difference  Between  MM5 

Forecast  Data  and  Observation  Data  for  the  Period  of  January-March,  2002  31 

Appendix  B.  Statistical  Diagrams  Displaying  the  Difference  Between  MM5 

Forecast  Data  and  Observation  Data  for  the  Period  of  April-June,  2002  43 

Appendix  C.  Statistical  Diagrams  Displaying  the  Difference  Between  BFM 

Forecast  Data  and  Observation  Data  for  the  Period  of  January-March,  2002  55 

Appendix  D.  Statistical  Diagrams  Displaying  the  Difference  Between  BFM 

Forecast  Data  and  Observation  Data  for  the  Period  of  April-June,  2002  67 


List  of  Figures 


Figure  1.  Geographical  extents  of  the  four  computational  domains . 

Figure  2.  Terrain  contours  of  MM5  domain  3 . 

Figure  3.  Terrain  contours  of  MM5  domain  4 . 

Figure  4.  Terrain  contours  of  the  BFM  model  domain  (2.5  km  grid  increments) . 

Figure  5.  Terrain  contours  of  the  BFM  model  domain  (5  km  grid  increments) . 

Figure  6.  Difference  between  MM5  forecast  and  observation:  temperature 

(domain  3) . 

Figure  7.  Difference  between  MM5  forecast  and  observation:  temperature 

(domain  4) . 

Figure  8.  Difference  between  MM5  forecast  and  observation:  dew  point  temperature 

(domain  3) . 

Figure  9.  Difference  between  MM5  forecast  and  observation:  temperature 

(domain  4) . 

Figure  10.  Difference  between  MM5  forecast  and  observation:  wind  vector-x 

component  u  (domain  3) . 

Figure  11.  Difference  between  MM5  forecast  and  observation:  wind  vector-x, 

component  u  (domain  4) . 

Figure  12.  Difference  between  MM5  forecast  and  observation:  wind  vector-y, 

component  v  (domain  3) . 

Figure  13.  Difference  between  MM5  forecast  and  observation:  wind  vector-y, 

component  v  (domain  4) . 

Figure  14.  Difference  between  MM5  forecast  and  observation:  wind  speed 

(domain  3) . 

Figure  15.  Difference  between  MM5  forecast  and  observation:  wind  speed 

(domain  4) . 


..5 

..6 

..7 

..8 

..9 

16 

16 

17 

17 

.18 

.18 

.19 

.19 

.20 

.20 


IV 


Figure  16.  Time  series  of  statistical  parameters  for  temperature 

. 23 

(April-June  2002,  MM5  domain  4; . 

Figure  17.  Time  series  of  statistical  parameters  of  temperature 

. 23 

(April-June  2002,  BrM  2. 5  Km  grid  increment; . 

Figure  18.  Time  series  of  statistical  parameters  for  dew  point  temperature 

. 24 

(April-June  2002,  MMj  domain  4; . 

Figure  19.  Time  series  of  statistical  parameters  for  temperature 

. 24 

(April-June  2002,  BrM  2.5  Km  grid  increment; . 

Figure  20.  Time  series  of  statistical  parameters  for  wind  speed 

. 25 

(Apnl-June  2002,  MMj  domain  o; . • . . . 

Figure  21.  Time  series  of  statistical  parameters  for  wind  speed 

. 25 

(April-June  2002,  BFM  2.5  grid  increment; . 

Figure  22.  Time  series  for  RMSVE,  MWDDF,  and  RMSE  of  wind  direction 

. 26 

difference  (Apnl-June  200 2,  mmo  aomam  . 

Figure  23.  Time  series  for  RMSVE,  MWDDF  and  RMSE  of  wind  direction 

difference  (April-June  2002,  BFM  2.5  km  grid  increment) . 

. 26 

List  of  Tables 


Table  1.  Portion  of  the  daily  data . . . 

Table  2.  Dates  of  archived  data  sets  and  model  calculations  for  24-h  forecasting . 

Table  3.  Statistical  parameters  calculated  for  surface  temperature . 

Table  4.  Statistical  parameters  calculated  for  wind  vector . 

Table  5.  Statistical  parameters  of  temperature  for  different  model  domains  (January- 

March  2002) . 

Table  6.  Statistical  parameters  of  dew  pt  temperature  for  different  model  domains 

(January-March  2002) . . . . . 

Table  7.  Statistical  parameters  of  wind  vector  for  different  model  domains 

(January-March  2002) . * . . . 

Table  8.  Statistical  parameters  of  temperature  for  different  model  domains 

(April- June  2002) . . 

Table  9.  Statistical  parameters  of  dew  point  temperature  for  different  model  domains 

(April-June  2002) . 

Table  10.  Statistical  parameters  of  wind  vector  for  different  model  domains 

(April-June  2002) . . . . . 


10 

11 

14 

,15 

.21 

.21 

.21 

.21 

,.22 


,22 


Executive  Summary 


The  forecast  skills  of  the  Penn  State/National  Center  for  Atmospheric  Research 
Mesoscale  Model  version  5  (MM5),  and  the  Battlescale  Forecast  Model  (BFM)  were 
statistically  evaluated  by  comparing  the  model  forecasting  data  with  surface  observation 
data  obtained  from  the  University  of  Utah  Mesowest  cooperative.  The  evaluation  study 
was  made  for  two  different  periods:  January-March  2002,  and  April-June  2002. 

The  MM5,  with  quadruple-nests  of  67.5, 22.5, 7.5,  and  2.5  km  grid  increments,  all  of 
which  have  51x51  grid  points  centered  near  Salt  Lake  City,  UT,  was  set  for  computation 
on  the  U.S.  Army  High  Power  Computer.  The  forecast  data  of  domains  3  and  4,  with  grid 
increments  of  7.5  and  2.5  km  (respectively),  are  compared  with  the  observation  data  in 
each  domain.  The  BFM  forecasting  calculations  were  made  separately  over  two  model 
domains  with  grid  increments  of  5  and  2.5  km,  both  centered  on  Salt  Lake  City,  UT. 

The  major  findings  of  this  study  include  the  following: 

1 .  There  are  no  significant  statistical  differences  in  forecast  results  between  MM5 
domains  3  and  4.  Likewise,  the  forecast  results  of  the  BFM  for  the  two  grid 
increments  of  5  and  2.5  km  are  statistically  similar. 

2.  Both  MM5  and  the  BFM  produce  better  forecast  results  for  surface  temperature 
than  for  dew  point  temperature,  and  have  difficulties  producing  good  forecast 
results  for  wind  vectors. 

3.  For  all  of  the  surface  meteorological  parameters,  MM5  produced  statistically 
better  forecast  results  than  the  BFM  over  the  complex  terrain  of  Utah. 

4.  Both  MM5  and  the  BFM  showed  better  forecasting  statistics  for  the  period  of 
April-June  2002  than  for  the  period  of  January-March  2002. 
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1.  Introduction 


The  Battlescale  Forecast  Model  (BFM),  developed  at  the  U.S.  Army  Research  Laboratory 
(ARL),  has  been  extensively  used  to  make  short-range  forecasts  of  atmospheric 
conditions  as  a  component  of  the  Integrated  Meteorological  System  (IMETS).  The  BFM 
uses,  for  prognostic  calculation,  a  hydrostatic  model  called  the  Higher  Order  Turbulence 
Model  for  Atmospheric  Circulation  (HOTMAC),  which  was  developed  by  Yamada  and 
Bunker  (1989).  The  HOTMAC  was  used  for  prognostic  calculation  because  of  its 
numerical  stability,  which  allows  operational  forecasting  calculations  to  be  made  without 
failure.  HOTMAC  was  also  chosen  because  of  its  short  computational  time  requirement. 
A  24-h  forecast  calculation  over  a  battlescale  area,  such  as  500x500  km  ,  could  be 
completed  by  BFM  within  one  hour  on  a  workstation  computer  having  a  single  processor. 

With  advancements  in  both  computer  hardware  and  software,  such  as  the  availability  of  a 
parallel  computer  with  distributed  memory  and  Message  Passage  Interface  (MPI), 
applications  like  non-hydrostatic  models  with  many  different  physics  options  are 
becoming  possible.  Thus,  it  is  important  to  evaluate  how  a  non-hydrostatic  model,  such 
as  the  Pennsylvania  State  University/National  Center  for  Atmospheric  Research  (NCAR) 
Mesoscale  Model  version  5  (MM5),  performs  over  areas  of  complex  terrain.  For 
comparison,  the  BFM  is  also  evaluated  in  this  study. 

In  a  previous  study,  the  forecast  skills  of  the  BFM  were  compared  to  those  of  MM5  by 
applying  the  models  to  the  domain  of  White  Sands  Missile  Range  (WSMR),  NM,  which 
covers  an  area  of  167x167  km,  51x51  grid  points  with  grid  spacing  of  3.33  km  (Henmi, 
2000).  Meteorological  parameters  forecasted  by  the  models  were  compared  with 
observed  data.  The  comparison  study  showed  that  the  forecast  skills  of  the  BFM  are 
comparable  to  those  of  the  MM5.  Surface  temperature  forecasted  by  both  the  BFM  and 
MM5  agreed  well  with  observed  values.  Both  the  BFM  and  the  MM5  showed  difficulties 
for  forecasting  the  relative  humidity.  In  forecasting  wind  parameters,  both  models  tended 
to  predict  wind  speed  less  than  that  observed,  but  BFM  calculations  produced  lower  wind 
speed  than  the  MM5.  The  BFM  resulted  in  better  forecasts  than  the  MM5  for  wind 
direction. 

In  operational  mode  on  the  IMETS,  the  BFM  has  been  extensively  used  over  the  model 
domain  of  500x500  km2  with  grid  spacing  of  10  km.  A  statistical  evaluation  of  the  BFM 
in  operational  mode  was  conducted  for  cases  during  a  30-day  period  over  three  different 
model  domains  (Colorado,  Washington  and  Florida),  which  had  different  terrain 
complexities  and  climates  (Henmi,  2000).  The  model  calculations  were  initialized  with 
three  different  sets  of  initial  conditions: 
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1 .  NOGAPS  +  upper  air  data  +  surface  data 

2.  NOGAPS  +  upper  air  data 

3.  NOGAPS 

Forecast  data  for  24-h  periods  were  statistically  compared  with  surface  observation  data 
by  calculating  parameters  such  as  mean  difference  (MD),  absolute  difference  (AD),  root 
mean  square  error  (RMSE),  root  mean  square  vector  error  (RMSVE),  and  correlation 
coefficient  (CC). 

For  all  three  model  domains,  the  temperature  fields  of  the  BFM  were  statistically  better 
when  initialized  with  (1)  and  (2)  than  when  initialized  with  (3).  For  Colorado  and 
Washington  model  domains,  the  BFM  showed  clear  tendencies  of  forecasting  dew  point 
temperature  lower  than  those  observed  throughout  the  24-h  forecast  period.  However,  for 
the  Florida  model  domain,  forecasts  of  dew  point  temperature  were  higher  than  observed. 

The  three  different  types  of  initialization  data  did  not  produce  significantly  different  wind 
fields  throughout  the  24-h  forecast  period.  The  value  of  MD  for  wind  speed  is  in  the 
range  of  0  to  1  m/sec.  The  values  of  AD  are  also  between  0  and  1  m/sec  for  the  three 
model  domains  throughout  the  24-h  forecast  period. 

For  the  Colorado  and  Washington  model  domains,  where  the  terrain  is  more  complex 
than  in  Florida,  the  use  of  the  BFM  improves  temperature  forecasts  over  those  of  the 
Navy  Operational  Global  Atmospheric  Prediction  System  (NOGAPS)  and  the  Navy 
Operational  Regional  Atmospheric  Prediction  System  (NORAPS).  For  the  Florida 
domain,  no  significant  improvements  in  temperature  forecasts  are  found.  Similarly,  the 
BFM  produces  better  wind  fields  than  NOGAPS  and  NORAPS  over  Colorado  and 
Washington. 

In  the  recent  study  of  an  objective  multiyear  evaluation  of  the  University  of  Washington 
real-time  MM5  forecasts,  in  which  forecasts  from  the  36, 12,  and  4  km  domains  were 
verified  at  surface  observation  locations  over  western  Washington  state,  Mass  et  al. 
states:  “Reduction  from  36  to  12  km  grid  increments  allows  the  definition  of  the  major 
topographic  features  of  the  region  and  their  corresponding  atmospheric  circulations, 
resulting  in  a  beneficial  effect  on  the  evaluation.  However,  there  are  only  small 
improvements  in  verification  statistics  as  grid  spacing  decreased  from  12  to  4  km.” 
(2002). 

The  purpose  of  this  report  is  to  describe  the  results  of  a  statistical  evaluation  of  surface 
forecast  data  from  MM5  domains  with  grid  increments  of  7.5  and  2.5  km,  and  from  BFM 
domains  with  grid  increments  of  5  and  2.5  km,  all  of  which  are  centered  near  Salt  Lake 
City,  UT,  and  use  Mesowest  data.  The  ARL  High  Power  Computer  (HPC)  parallel 
computer  is  extensively  used  for  this  study. 
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2.  Description  of  the  Models  and  Model  Domains 


2.1  MM5 

The  model  used  in  this  study  is  the  distributed-memory  version  of  the  PSU-NCAR  MM5. 
The  source  code  of  MM5  was  obtained  from  die  MM5  Home  page  (2001). 

The  MM5  is  based  on  non-hydrostatic  dynamics  and  features  multiple-nest  capabilities 
and  many  physics  options.  Details  of  this  modeling  system  can  be  found  in  Dudhia 
(1993),  Grell  et  al.  (1994)  and  Warner  et  al.  (1992). 

For  the  present  study,  the  following  physics  options  are  employed: 

1 .  Planetary  Boundary  Layer  (PBL).  The  PBL  technique  that  is  used  in  the  Medium- 
Range  Forecast  (MRF)  model  of  the  National  Center  for  Environmental 
Prediction  (NCEP)  by  Hong  and  Pan  (1996). 

2.  Precipitation  parameterization.  A  simple  treatment  of  cloud  microphysics  based 
on  Dudhia  (1989),  in  which  both  ice  and  liquid  phases  are  permitted  for  cloud  and 
precipitation,  but  mixed  phases  are  not  permitted. 

3.  Cumulus  parameterization.  GrelTs  scheme  that  is  based  on  rate  of  destabilization 
or  quasi-equilibrium,  simple  single-cloud  scheme  with  updraft  and  downdraft 
fluxes,  and  compensating  motion  determining  heating  and  moistening  profile 
(1994). 

4.  Radiation  parameterization.  Dudhia’s  scheme  in  which  long  wave  and  short  wave 
radiation  interact  with  the  clear  atmosphere,  clouds,  precipitation,  and  the  ground 
(1989). 

5 .  Ground  temperature  scheme.  Multilayer  soil  temperature  model. 

The  MM5  quadruple-nested  computational  grids  are  depicted  in  figure  1.  All  four 
computational  domains  have  a  mesh  size  of  55x55  grid  points,  and  grid  increments  of 
67.5, 22.5,  7.5,  and  2.5  km,  respectively,  for  domains  1, 2,  3,  and  4.  For  the  present  study, 
the  forecast  fields  only  for  domains  3  and  4  are  compared  with  observed  data. 
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Figure  1.  Geographical  extents  of  the  four  computational  domains. 


Terrain  height  contours  of  domains  3  and  4  are  shown,  respectively,  in  figures  2  and  3. 
Terrain,  vegetation/land  use,  and  soil  type  data  for  the  model  domains  are  obtained  from 
data  archived  in  the  MM5  Internet  site. 
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Figure  2.  Terrain  contours  of  MM5  domain  3. 
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Figure  3.  Terrain  contours  of  MM5  domain  4. 


2.2  BFM 

The  BFM  is  an  operational  mesoscale  forecast  model  developed  by  the  ARL  (Henmi  and 
Dumais,  1998)  at  WSMR,  NM.  It  has  been  extensively  used  to  make  short-range 
forecasts  of  atmospheric  conditions  as  a  component  in  the  IMETS.  The  BFM  uses,  for 
prognostic  calculation,  the  HOTMAC,  developed  by  Yamada  and  Bunker  (1989).  In 
operational  mode  on  the  IMETS,  the  BFM  has  been  extensively  used  over  a  model 
domain  of  500x500  km  with  grid  increments  of  1 0  km.  So  far,  there  has  been  no 
evaluation  study  of  the  BFM  over  the  domain  with  grid  increment  smaller  than  5  km  for 
an  extended  period.  In  this  study,  the  BFM  was  used  over  two  domains  with  2.5  and  5  km 
grid  increments,  centered  at  Salt  Lake  City,  UT. 

Figure  4  shows  the  terrain  contours  of  the  2.5  km  grid  increment  domain.  Figure  5  shows 
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the  terrain  contours  of  the  5  km  grid  increment  domain.  An  inner  squared  area  of  5 1x5 1 
grid  points  with  125  km2  and  250  km2  are  used  for  the  model  calculations,  respectively, 
for  figures  4  and  5.  In  these  figures,  the  inner,  squared  areas  represent  the  BFM  model 
domains.  The  spots  denoted  by  “N”  are  the  locations  of  the  NOGAPS  grid  points  that  are 
used  for  initial  and  time-dependent  boundary  values,  and  the  spots  denoted  by 
represent  the  Mesowest  data  points.  For  model  evaluation,  data  within  the  squared  area  is 
used. 


Figure  4.  Terrain  contours  of  the  BFM  model  domain 
(2.5  km  grid  increments). 
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Figure  5.  Terrain  contours  of  the  BFM  model  domain 
(5  km  grid  increments). 


3.  Meteorological  Data 


There  is  a  large  amount  of  surface  observation  data  available  for  the  model  domains  that 
are  shown  in  figures  4  and  5.  The  data  is  freely  available  from  the  University  of  Utah 
Mesowest  cooperative. 

A  Mesowest  data  file,  called  “total.dat,”  contains  data  in  15-min  intervals  for  a  24-h 
period  for  the  western  United  States.  A  detailed  description  of  Mesowest  data  can  be 
found  in  Horel  et  al.  (2002). 

A  daily  data  file  that  contains  data  only  for  the  even  hours  of  a  24-h  period  (starting  at 
00:00  UTC  (universal  time  coordinate))  was  created  for  the  areas  to  be  studied.  Table  1 
displays  a  portion  of  the  daily  data  file.  Table  1  shows  the  following  (from  left  to  right): 
station  name,  day,  hour,  minute,  latitude,  longitude,  wind  direction,  wind  speed  (knot), 
temperature  (°F),  dew  point  temperature  (°F),  relative  humidity  (%),  and  24-h 
accumulated  precipitation  (in  this  example  -9999.0  zero  or  no  report).  This  data  was 
compared  to  hourly  model  forecast  data. 
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Table  1.  Portion  of  the  daily  data. 


BGRUT 

2 

516 

0 

0 

40.93 

-112 . 56 

90.00 

BR4 

2 

516 

0 

0 

41.44 

-112.16 

148.00 

DVB 

2 

516 

0 

0 

40.62 

-111.48 

248.00 

EMPUT 

2 

516 

0 

0 

40.61 

-111.53 

245.00 

FPK 

2 

516 

0 

0 

41.03 

-111.84 

323.00 

FWP 

2 

516 

0 

0 

40.66 

-112.20 

244.00 

GNI 

2 

516 

0 

0 

41.33 

-112.85 

96.00 

HAT 

2 

516 

0 

0 

41.07 

-112.59 

42.00 

LDS1 

2 

516 

0 

0 

40.77 

-111.89 

359.00 

LMR 

2 

516 

0 

0 

41.06 

-112.89 

104.00 

LMS 

2 

516 

0 

0 

41.70 

-112.85 

148.00 

MBY 

2 

516 

0 

0 

40.61 

-111.48 

282.00 

PCB 

2 

516 

0 

0 

40.65 

-111.51 

347.00 

PCS 

2 

516 

0 

0 

40.65 

-111.52 

266.00 

PRP 

2 

516 

0 

0 

41.26 

-112.44 

341.00 

SB1 

2 

516 

0 

0 

41.19 

-111.84 

312.00 

SNH 

2 

516 

0 

0 

40.55 

-111.85 

340.00 

SNI 

2 

516 

0 

0 

41.20 

-111.86 

20.00 

SNV 

2 

516 

0 

0 

40.62 

-111.51 

269.00 

SNX 

2 

516 

0 

0 

41.04 

-112.23 

48.00 

SNZ 

2 

516 

0 

0 

40.88 

-111.87 

339.00 

TPC 

2 

516 

0 

0 

40.43 

-111.71 

192.00 

WBU 

2 

516 

0 

0 

40.71 

-111.56 

271.00 

WCR 

2 

516 

0 

0 

40.55 

-111.32 

261.00 

KMS 

2 

516 

0 

0 

40.56 

-111.13 

253.00 

SKY 

2 

516 

0 

0 

39.58 

-111.25 

249.00 

UT1 

2 

516 

0 

0 

41.20 

-111.11 

225.00 

UT12 

2 

516 

0 

0 

40.64 

-111.90 

0.00 

UT20 

2 

516 

0 

0 

40.75 

-111.90 

337.50 

UT23 

2 

516 

0 

0 

40.72 

-111.90 

0.00 

UTS 

2 

516 

0 

0 

40.71 

-111.80 

270.00 

UT7 

2 

516 

0 

0 

40.48 

-111.90 

0.00 

UT9 

2 

516 

0 

0 

40.68 

-112.26 

22.50 

SND 

2 

516 

0 

0 

40.37 

-111.59 

167.00 

QSA 

2 

516 

0 

0 

40.83 

-112.01 

5.00 

KEM 

2 

516 

0 

0 

41.76 

-110.58 

270.00 

LGP 

2 

516 

0 

0 

41.71 

-111.71 

288.00 

CASD 

2 

516 

0 

0 

39.22 

-111.02 

257.40 

ELMO 

2 

516 

0 

0 

39.40 

-110.81 

286.60 

FERR 

2 

516 

0 

0 

39.09 

-111.09 

359.30 

GUNN 

2 

516 

0 

0 

39.16 

-111.81 

333.20 

LITW 

2 

516 

0 

0 

39.32 

-111.33 

339.80 

MOLE 

2 

516 

0 

0 

39.06 

-111.14 

202.50 

CUPC 

2 

516 

0 

0 

40.36 

-111.90 

333.40 

CUPE 

2 

516 

0 

0 

40.49 

-111.47 

255.40 

CUPF 

2 

516 

0 

0 

40.22 

-111.11 

273.60 

CUPG 

2 

516 

0 

0 

40.37 

-111.03 

355.30 

CUPI 

2 

516 

0 

0 

40.56 

-110.70 

354.40 

MSX01 

2 

516 

0 

0 

40.72 

-111.86 

325.00 

BBN 

2 

516 

0 

0 

40.89 

-111.85 

330.00 

ARAU1 

2 

516 

0 

0 

40.59 

-113.02 

40.00 

BRAU1 

2 

516 

0 

0 

40.88 

-110.83 

90.00 

7.80 

62.00 

35.50 

37.00-9999.00 

0.90 

65.00 

27.40 

24.00-9999.00 

2.60 

56.00 

19.70 

24.00-9999.00 

4.30 

45.00 

20.30 

37.00-9999.00 

13.00 

38.00 

8.50 

29.00-9999.00 

3.50 

43.00 

17.20 

35.00-9999.00 

8.70 

62.00 

29.40 

29.00-9999.00 

7.00 

62.00 

23.80 

23.00-9999.00 

3.50 

63.00 

17.50 

17.00-9999.00 

0.00 

60.00 

22.10 

23.00-9999.00 

2.60 

67.00 

20.80 

17.00-9999.00 

9.60 

48.00 

16.50 

28.00-9999.00 

1.70 

60.00 

24.10 

25.00-9999.00 

1.70 

55.00 

20.80 

26.00-9999.00 

6.10 

53.00 

10.60 

18.00-9999.00 

3.50 

55.00 

13.50 

19.00-9999.00 

4.30 

65.00 

26.30 

23.00-9999.00 

0.90 

50.00 

15.60 

25.00-9999.00 

0.90 

52.00 

20.70 

29.00-9999.00 

4.30 

62.00 

27.70 

27.00-9999.00 

0.00 

66.00 

25.00 

21.00-9999.00 

4.30 

52.00 

23.10 

32.00-9999.00 

4.30 

61.00 

25.00 

25.00-9999.00 

7.80 

57.00 

16.30 

20.00-9999.00 

3.50 

57.00 

18.50 

22.00-9999.00 

8.70 

52.00 

16.30 

24.00-9999.00 

10.50 

56.10 

16.20 

20.00-9999.00 

5.80 

66.00 

21.90 

18.00-9999.00 

6.80 

66.70 

20.80 

17.00-9999.00 

7.20 

67.60 

20.70 

16.00-9999.00 

6.00 

64.90 

17.20 

15.00-9999.00 

12.60 

65.10 

19.90 

17.00-9999.00 

15.60 

63.10 

23.40 

22.00-9999.00 

7.00 

55.00 

19.80 

25.00-9999.00 

7.00 

63.00 

22.50 

21.00-9999.00 

16.50 

54.00 

25.60 

33.00-9999.00 

10.40 

36.00 

14.60 

41.00-9999.00 

8.80 

74.10 

28.20 

18.20-9999.00 

5.90 

77.00 

20.60 

12.10-9999.00 

11.60 

68.80 

17.70 

14.10-9999.00 

11.20 

74.60 

27.00 

17.00-9999.00 

7.20 

67.50 

33.90 

28.80-9999.00 

10.50 

70.90 

18.20 

13.30-9999.00 

6.30 

67.20 

27.40 

22.30-9999.00 

6.30 

65.80 

25.90 

22.00-9999.00 

11.70 

54.60 

31.00 

40.40-9999.00 

7.30 

54.40 

30.30 

39.50-9999.00 

0.00 

55.80 

14.60 

19.40-9999.00 

5.60 

69.70 

25.60 

19.00-9999.00 

2.60 

63.00 

17.50 

17.00-9999.00 

6.10 

64.00 

13.00 

14.00-9999.00 

4.30 

54.00 

19.00 

26.00-9999.00 

NOGAPS  data,  which  is  available  for  every  1°  grid  point  for  the  entire  world,  is  used  for 
initialization  and  time-dependent  lateral  boundary  conditions  for  both  the  MM5  and  the 
BFM.  The  forecast  data  for  the  periods  of  0, 12,  and  24-h,  initialized  at  00:00  UTC,  were 
obtained  at  13  pressure  levels  from  1000  mb  to  100  mb.  The  data  was  obtained  through 
the  Defense  Modeling  and  Simulation  Office,  Master  Environmental  Library  Web  site. 
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Upper-air  sounding  data  was  obtained  from  the  University  of  Wyoming  Web  site. 

For  this  study,  these  three  different  data  sets  for  the  winter  and  spring  of  2002  were 
collected,  and  model  calculations  for  24-h  forecasting  were  performed  for  both  seasons 
(Table  2). 


Table  2.  Dates  of  archived  data  sets  and  model  calculations  for  24-h  forecasting. 


Winter,  2002 

Spring,  2002 

January  15,  22, 23,  24,  25,  28,  29,  30 

April  17,  22, 23, 24, 30 

February  4,  5,  6, 7,  8, 11, 12, 13, 14, 19, 
20,21,22, 26, 27,28 

May  1, 2,  6, 8, 13, 14,  16, 20, 21, 30  31 

March  4,  5,  6,  7,  8,  11,  12,  14 

June  3, 4,  5, 10,11, 17,18, 19,  20 

4.  Forecast  and  Evaluation  Methods 


4.1  Forecast  Method 

4.1.1  MM5 

The  standard  method  of  MM5  initialization  (REGRID)  is  used,  which  is  discussed  in  the 
MM5  user’s  manual  (MM5  Home  Page,  2001).  GRIB-formatted  NOGAPS  data  on 
pressure  levels  is  read  and  interpolated  to  the  MM5  grid.  For  the  MM5  calculations,  only 
NOGAPS  data  is  used.  The  program,  INTERP,  conducts  vertical  interpolation,  diagnostic 
computation  and  data  reformatting  to  generate  model  initial  and  lateral  boundary 
conditions,  and  a  lower  boundary  condition.  The  data  sets  are  generated  separately  for 
four  different  nested  grids. 

4.1.2  BFM 

From  the  NOGAPS  data,  horizontal  wind  vector  components,  temperature,  dew  point 
temperature  and  geopotential  height  at  13  different  pressure  levels  for  the  forecast  periods 
of  0, 12,  and  24-h  are  interpolated  to  the  81x81  grid  with  grid  increments  of  2.5  or  5  km 
at  each  pressure  level  for  the  above  forecast  periods.  The  data  is  then  vertically 
interpolated  from  the  pressure  levels  to  the  BFM’s  height  levels  to  produce  4-d  fields  of 
the  input  data  for  initialization  and  lateral  boundary  conditions.  In  this  study,  all  model 
calculations  were  initiated  at  00:00  UTC.  The  0-h  NOGAPS  data  is  composed  with  upper 
air  sounding  data  obtained  at  Salt  Lake  City.  The  details  of  the  initialization  method  are 
described  in  Henmi  and  Dumais  (1998). 
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4.2  Evaluation  Method 


4.2.1  Data  Comparison  Method 

After  the  forecast  calculation  is  completed,  the  following  bilinear  interpolation  is 
conducted  to  obtain  model  forecast  data  at  the  surface  observation  locations. 

Suppose  a  surface  observation  location  (x*  and  y’)  is  surrounded  by  four  model  grid 
points.  An  interpolated  value,  cp’,  of  an  arbitrary  variable,  9  at  (x\  y’),  is  calculated  using 
a  bilinear  interpolation  method,  such  as: 

cpi=cp(i,  j)+(x’  -x)[(p(i+l,  j)  -  <p(i>  j)] 

92=  9(i,  j+1)  +(x’  -x)[9(i+l,  j+1)  -  9(i»  j+1)] 

9’(x’,y’)=9i  +  (y’-y)[92  -  <Pi]  ^ 


where 

(i,  j)  is  the  southwest  grid  point  of  the  four  grid  points  surrounding  a  surface 
observation  location  (x\  y’),  and  (x,  y)  is  the  location  for  the  grid  point  (1,  j): 
9  (i,  j)  is  an  arbitrary  variable  at  (x,  y). 


4.2.2  Statistical  Parameters 

The  following  statistical  parameters  between  model  forecast  data  and  surface  observation 
data  are  calculated  using  the  data  available  in  the  model  domains.  The  parameters  are 
calculated  hourly  for  the  entire  period.  Statistical  parameters  are  calculated  for 
temperature,  relative  humidity,  wind  speed  and  horizontal  wind  vector  components  u  and 

v. 


Mean  Difference  (MD): 


m  n 

Xo,i,P 


MD  = 


M  i= 1 


mn 


(2) 

where  . 

the  subscript  o  represents  observation  and  p  represents  prediction.  The 
subscript  i  represents  the  ith  surface  station,  and  the  subscript  y  represents 
the/*  forecast  day.  n  is  the  number  of  surface  stations,  and  m  is  the  total 
number  of  forecast  days. 

A  nonzero  MD  indicates  bias.  For  instance,  if  the  MD  value  is  positive,  it 
indicates  that  the  model  tends  to  over-forecast. 
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Mean  Absolute  Difference  (AD): 


m  n 

Xp,i,j  Xo,i,j 


AD  = 


j= i  w 


Root  Mean  Square  Error  (RMSE): 


RMSE 


( Xo,i,j  Xo,i,P 

7=1  «=1 _ 


Better  agreements  between  observation  and  forecast  are,  in  general,  related  to 
smaller  values  of  AD  and  RMSE. 

Root  Mean  Square  Vector  Error  (RMSVE): 


RMSVE  - 


tri  ti  - 

YlMo.tj-UpJ  +(Vo,i,rvPJ  ] 

J=Li= 1 _ - — 


This  parameter  measures  die  differences  of  both  wind  speed  and  direction.  Good 
agreements  of  wind  vectors  are  related  to  small  values  of  the  RMSVE. 


Correlation  Coefficient  (CC): 


cc  = 


m  n 

II  y^yM 


Im  n  0  rn  ri  2 

5^„,iit,, 


where 


yrJ,rx  xp 


y0,ij  Xojj  x0 

xn  and  xD  are  the  means  of  observed  and  forecasted  values,  respectively. 


13 


5.  Results 


The  following  section  presents  statistical  parameters  that  were  calculated  between  model 
forecast  data  and  surface  observation  data.  For  this  study,  all  of  the  MM5  forecast 
calculations  are  done  with  a  6-h  spin-up  period.  So  for  the  MM5,  forecast  data  is  only 
available  between  06:00  and  24:00  UTC.  For  the  BFM,  24-h  forecast  data  between  00:00 
and  24:00  UTC  is  available. 


5. 1  Statistical  Parameters  of  Daily  Data 

Table  3  shows  the  statistical  parameters  (CC,  MD,  and  RMSE)  of  surface  temperature 
calculated  for  the  daily  data  from  the  period  of  January-March  2002.  Table  4  shows  the 
statistical  parameters  (CC,  MD  and  the  mean  wind  direction  difference  between  model 
and  observation  data  (MWDDF))  of  wind  vector  calculated  for  the  daily  data  from  the 
period  of  January-March  2002.  The  results  that  are  displayed  in  Tables  3  and  4  are  for 
MM5  domain  4  and  the  BFM  of  the  grid  increment  of  2.5  km. 


Table  3.  Statistical  parameters  calculated  for  surface  temperature  (continued). 


3-12 

- -  r _ 

.77 

-.6 

2.6 

.40 

-2.3 

4.9 

3-14 

.87 

-2.1 

3.3 

.69 

3.3 

4.6 

Table  4.  Statistical  parameters  calculated  for  wind  vector. 


MM5 

BFM 

Date 

CC 

RMSVE 

MWDDF 

CC 

RMSVE 

MWDDF 

1-15 

.20 

3.6 

56.9 

.15 

4.3 

64.1 

1-22 

.11 

5.6 

43.1 

-.18  ' 

7.3 

55.9 

1-23 

.21 

2.6 

65.6 

.00 

2.9 

62.9 

1-24 

.17 

3.3 

64.8 

.00 

3.2 

73.6 

1-25 

.39 

4.4 

55.0 

.13 

3.4 

55.0 

1-28 

.20 

6.3 

52.2 

.11 

5.8 

50.7 

1-29 

.29 

2.8 

46.3 

-.01 

2.6 

42.6 

1-30 

.00 

4.2 

62.8 

.03 

2.9 

52.8 

2-04 

.40 

2.3 

34.5 

.07 

2.3 

41.8 

2-05 

.06 

2.9 

58.6 

-.04 

2.8 

45.6 

2-06  1 

.26 

2.3 

70.5  . 

.00 

2.6 

70.0 

2-07 

.24 

3.9 

57.2 

.03 

2.8 

62.3 

2-08 

.23 

7.0 

45.2 

.07 

5.7 

69.8 

2-11 

.31 

3.1 

54.3 

.09 

2.7  1 

55.0 

2-12 

.48 

2.8 

35.9 

.10 

3.5 

37.6 

2-13 

.10 

3.7 

60.7 

.03 

4.2 

79.4 

2-14 

.51 

3.4 

42.2 

.18 

4.0 

61.2 

2-19 

.29 

3.0 

49.3 

.02 

4.0 

75.0 

2-20 

.31 

4.7 

60.8 

-.03 

4.9 

77.5 

2-22 

.50 

3.4 

48.8 

-.03 

3.5 

70.0 

2-26 

.23 

3.6 

57.3 

-.08 

4.5 

67.4 

2-27 

.21 

4.0 

31.1 

.08 

4.7 

53.1 

2-28 

.41 

4.5 

50.8 

.17 

4.6 

70.3 

3-04 

.51 

2.6 

66.4 

.02 

3.4 

66.2 

3-05 

.26 

4.0 

45.2 

.10 

3.2 

61.0 

3-07 

.19 

5.7 

43.2 

.03 

4.0 

65.7 

3-11 

.45 

3.29 

46.7 

-.05 

3.9 

59.8 

3-12 

.50 

4.0 

32.1 

.01 

9.8 

73.2 

3-14 

.12 

4.0 

28.6 

-.13 

4.7 

45.8 

The  values  of  daily  statistical  parameters  for  both  temperature  and  wind  vary 
considerably  between  days.  For  surface  wind  forecasts,  the  CC  values  for  MM5  are 
greater  than  those  for  the  BFM,  and  the  MWDDF  values  for  MM5  are  smaller  than  those 
for  the  BFM  throughout  the  period  (Table  4).  These  results  indicate  that  MM5  performed 
better  than  the  BFM  in  wind  forecasts  over  the  complex  terrain  of  Utah.  However,  for 
surface  temperature  forecasts,  statistical  parameters  for  both  models  are  compatible 
(Table  3).  As  shown  in  the  appendix,  similar  results  are  obtained  between  MM5  domain  3 
and  the  BFM  with  5  km  grid  increments. 

5.2  MM5  Forecast  and  Observation  Data  Comparison 

Figures  6-15  are  scatter  diagrams  that  illustrate  the  difference  between  the  MM5  forecast 
data  and  the  observation  data  for  the  period  of  January-March  2002.  For  each  set  of 
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figures  that  follow,  MM5  domain  3  with  grid  increments  of  7.5  km  is  the  figure  displayed 
at  the  top  of  the  page,  and  MM5  domain  4  with  grid  increments  of  2.5  km  is  the  figure 
displayed  at  the  bottom. 

Diagrams  for  the  period  of  April- June  2002  can  be  found  in  Appendix  B.  Figures 
representing  the  BFM  for  the  periods  of  January-March  and  April- June  2002  are  included 
in  Appendixes  C  and  D. 


Observat i on 

Figure  6.  Difference  between  MM5  forecast  and 
observation:  temperature  (domain  3). 


Observat i on 

Figure  7.  Difference  between  MM5  forecast  and 
observation:  temperature  (domain  4). 
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Fore.ca.st 


Observa t i on 

Figure  8.  Difference  between  MM5  forecast  and 

observation:  dew  point  temperature  (domain  3). 


i - < - 1 - 1 — - r 

TD  CC> 


Observat i on 

Figure  9.  Difference  between  MM5  forecast  and 
observation:  temperature  (domain  4). 
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Observat \ on 

Figure  10.  Difference  between  MM5  forecast  and 
observation:  wind  vector-x  component  u 
(domain  3). 


Observat i on 

Figure  11.  Difference  between  MM5  forecast  and 

observation:  wind  vector-x,  component  u 
(domain  4). 
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SPEED  (M/S)  '  T 


Observa t i on 


Figure  14.  Difference  between  MM5  forecast  and 
observation:  wind  speed  (domain  3). 


SPEED  (M/S) 


Observa t i on 

Figure  15.  Difference  between  MM5  forecast  and 
observation:  wind  speed  (domain  4). 


Figures  6-15  were  obtained  by  plotting  all  of  the  data  accumulated  including  18  forecast 
hours  during  a  period  of  30  days  in  January-March  2002.  For  instance,  the  figure 
representing  the  temperature  of  MM5  domain  4  (figure  7)  contains  more  than  27,000 
points.  Similar  figures  for  the  BFM  can  be  found  in  Appendixes  C  and  D. 
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5.3  Statistical  Parameters  of  Entire  Forecasting  Periods 

Statistical  parameters  are  calculated  for  the  data  inclusive  of  all  forecast  hours  and  days 
during  the  two  seasons.  Tables  5, 6,  and  7  are  calculated  for  the  period  of  January-March 
2002,  and  Tables  8, 9,  and  10  are  calculated  for  the  period  of  April-June  2002.  Tables  5 
and  8  contain  CC,  MD,  AD,  and  RMSE  of  temperature  for  four  different  model  domains. 
Tables  6  and  9  contain  CC,  MD,  AD,  and  RMSE  of  dew  point  temperature.  Tables  7  and 
10  contain  the  statistical  parameters  for  wind:  CC,  MD,  and  AD  of  wind  speed,  and 
RMSVE  and  MWDDF. 


Table  5  Statistical  parameters  of  temperature  for  different  model  domains  (January-March  2002). 


Parameters 

MM5D3 

7.5  km  grid 

MM5D4 

2.5  km  grid 

BFM 

5  km  grid 

BFM 

2.5  km  grid 

CC 

.79 

.81 

.71 

.69 

MD 

.30 

-.02 

1.5 

1.3  ' 

AD 

3.3 

2.9  n 

3.7 

3.6 

RMSE 

4.3 

3.9 

4.9 

4.8 

#  of  Data 

43,192 

27,042 

56,300 

35,807 

Table  6.  Statistical  parameters  of  dew  pt  temperature  for  different  model  domains  (January-March  2002) 


Parameters 

MM5D3 

7.5  km  grid 

MM5D4 

2.5  km  grid 

BFM 

5  km  grid 

BFM 

2.5  km  grid 

CC 

.66 

.64 

.61 

.63 

MD 

-1.6 

-1.9 

-2.1 

-2.8 

AD 

4.5 

4.6 

4.6  1 

4.8 

RMSE 

5.7 

5.9 

5.6 

5.8 

#  of  Data 

42,864 

26,830 

57,181 

36,388 

Table  7.  Statistical  parameters  of  wind  vector  for  different  model  domains  (January-March  2002). 


Parameters 

MM5D3 

7.5  km  grid 

MM5 

2.5  km  grid 

BFM 

5  km  grid 

BFM 

2.5  km  grid 

CC  of  speed 

.37 

.42 

.09 

.09 

MD  of  speed 

.7 

.5 

-.4  1 

.0 

AD  of  speed 

1.9 

1.9 

1.9 

2.1  1 

RMSVE 

4.1 

4.1 

4.7 

5.0 

MWDDF 

51.2 

49.4 

62.7 

61.2 

#  of  Data 

37,177 

22,594 

47,706 

29,420 

Table  8.  Statistical  parameters  of  temperature  for  different  model  domains  (April-June  2002). 


Parameters 

MM5D3 

7.5  km  grid 

MM5 

2.5  km  grid 

BFM 

5  km  grid 

BFM 

2.5  km  grid 

CC 

.86 

.87  1 

.70 

.70 

MD 

-.62 

-.64 

-.5 

-.4 

AD 

3.3 

3.2 

4.5 

4.5 

RMSE 

4.2 

4.0 

6.1 

6.1 

#  of  Data 

19,338 

13,520 

19,636  1 
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Table  9.  Statistical  parameters  of  dew  point  temperature  for  different  model  domains  (April-June  2002). 


Parameters 

MM5D3 

7.5  km  grid 

MM5 

2.5  km  grid 

BFM 

5  km  grid 

BFM 

2.5  km  grid 

cc 

.56 

.53 

.33 

.32 

MD 

4.0 

3.8 

-2.3 

-2.5 

AD 

5.5 

5.4 

4.9 

5.0 

RMSE 

7.0 

6.9 

6.5 

6.6 

#  of  Data 

19,429 

13,573 

28,211 

19,757 

Table  10.  Statistical  parameters  of  wind  vector  for  different  model  domains  (April- June  2002). 


Parameters 

MM5D3 

7.5  km  grid 

MM5 

2.5  km  grid 

BFM 

5  km  grid 

BFM 

2.5  km  grid 

CC  of  speed 

.28 

.25 

.07 

.02 

MD  of  speed 

.37 

.50- 

-.2 

.1 

AD  of  speed 

2.0 

2.2 

2.3 

2.5 

RMSVE 

4.3 

4.4 

4.7 

5.0 

MWDDF 

57.1 

54.4 

59.5 

61.1 

#  of  Data 

17,077 

11,737 

24,354 

16,934 

From  the  data  displayed  in  Tables  5-10,  the  following  can  be  inferred: 

1 .  Both  MM5  and  the  BFM  yielded  better  forecasting  statistics  in  the  period  of  April- 
June  2002  than  in  the  period  of  January-March  2002.  This  may  be  caused  by  the 
following:  over  the  middle  latitude  area,  the  forcing  by  synoptic  weather  patterns  is 
more  dominant  than  the  forcing  by  local  surface  heating/cooling  in  cold  months  than 
in  warm  months.  Synoptic  weather  forcing  is  inputted  into  model  calculation  as  initial 
and  lateral  condition  values  from  the  NOGAPS  forecast  data,  and  local  forcing  effects 
are  calculated  by  surface  physics  routines  of  the  models. 

2.  There  are  no  significant  statistical  differences  between  MM5  domain  3  (grid 
increments  of  7.5  km)  and  domain  4  (grid  increments  of  2.5  km).  Similarly,  the 
statistical  parameters  for  the  BFM  of  grid  increments  5  km  and  2.5  km  are  similar. 
Reducing  the  grid  increment  does  not  produce  a  significant  improvement  of  forecast 
skills  for  either  MM5  or  the  BFM. 

3.  For  all  the  meteorological  parameters  of  temperature,  dew  point,  and  wind  vector,  the 
statistical  parameters  for  MM5  are  better  than  those  for  the  BFM.  Note  that  the 
amount  of  data  is  different  for  MM5  and  the  BFM. 

4.  Both  MM5  and  the  BFM  show  better  forecast  skills  for  temperature  than  they  show 
for  dew  point  temperature  and  wind  vector. 

5.  Over  the  complex  terrain  of  Utah,  both  MM5  and  BFM  show  low  forecast  skills  for 
wind  vector. 

5.4  Time  Series  of  the  Statistical  Parameters 

Every  statistical  parameter  is  calculated  at  each  forecast  hour  using  all  available  data  for 
the  period.  In  this  section,  figures  16-23  display  the  time  series  for  MM5  domain  4,  and 
the  BFM  of  2.5  km  grid  increment  for  the  period  of  April-June  2002.  For  MM5,  the  data 
for  the  time  period  of  06-23  h  are  plotted.  For  the  BFM,  the  data  is  plotted  from  00-24  h. 
Similar  figures  are  shown  in  Appendixes  A,  B,  C,  and  D  for  other  model  domains  and 
periods. 
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Figure  16.  Time  series  of  statistical  parameters  for  temperature  (April- June  2002,  MM5  domain  4). 


Forecast  time  <  hrs .  ) 


Figure  17.  Time  series  of  statistical  parameters  of  temperature  (April-June  2002,  BFM  2.5  km  grid 
increment). 


23 


domain  4). 


Figure  19.  Time  series  of  statistical  parameters  for  temperature  (April-June  2002,  BFM  2.5  km  grid 
increment). 
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Figure  21.  Time  series  of  statistical  parameters  for  wind  speed  (April-June  2002,  BFM  2.5  grid 
increment). 
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Figure  23.  Time  series  for  RMSVE,  MWDDF  and  RMSE  of  wind  direction  difference  (April- June 
2002,  BFM  2.5  km  grid  increment). 


From  the  information  displayed  in  figures  1 1-18,  the  following  can  be  seen: 

1 .  For  temperature,  both  the  MM5  and  the  BFM  forecast  data  produced  statistically 
good  agreement  with  observed  data,  with  the  AD  values  around  2  °C  and  the  CC 
greater  than  0.9. 

2.  Dew  point  temperatures  forecasted  by  MM5  tend  to  be  greater  than  the  dew  point 
temperatures  of  the  observed  data.  Dew  point  temperatures  forecasted  by  the 
BFM  tend  to  be  less  than  the  dew  point  temperatures  of  the  observed  data.  For 
MM5,  the  modeled  atmosphere  near  the  surface  tends  to  be  more  humid  than  the 
observed  atmosphere.  For  BFM,  the  modeled  atmosphere  near  the  surface  tends  to 
be  less  humid  than  the  observed  atmosphere.  The  forecast  skills  of  both  MM5  and 
the  BFM,  with  regards  to  dew  point  temperature,  are  statistically  less  than  those 
for  temperature. 

3.  The  AD  values  of  wind  speed  for  both  MM5  and  the  BFM  are  between  2  and  3 
m/sec,  and  the  RMSVE  values  are  between  4  and  5  m/sec.  In  the  previous  study 
(Henmi,  2000),  the  AD  values  of  three  different  climatological  areas  (Colorado, 
Washington,  and  Florida)  over  a  500x500  km2  area  with  grid  increment  of  10  km 
were  smaller  than  1  m/sec,  and  the  RMSVE  values  were  between  2  and  3  m/sec. 
Lower  statistical  skills  of  both  MM5  and  the  BFM  for  wind  vector  forecasts  over 
the  Utah  domains  in  this  study  may  be  attributed  to  the  fact  that  the  observed  data 
used  for  comparison  in  Henmi  (2000)  was  obtained  at  sites  such  as  airports,  where 
the  topography  surrounding  the  sites  is  flat.  Thus,  the  BFM  might  have  simulated 
wind  vector  fields  over  the  previous  domains  better  than  over  the  Utah  domains. 

4.  The  CC  values  of  MM5  (approximately  0.3)  and  of  the  BFM  (less  than  0.2),  and 
the  mean  wind  direction  differences  for  both  models  (greater  than  60°)  are 
statistically  poor. 

Similar  findings  can  be  found  in  the  statistics  of  MM5  domain  3  and  the  BFM  with  grid 
increments  of  5  km  (see  appendixes). 


6.  Conclusions 


Forecast  calculations  by  the  MM5  and  the  BFM  mesoscale  models  were  made  over  the 
model  domains  of  Utah  for  two  different  periods  (January-March,  April-June  2002). 

The  mesoscale  model,  MM5,  has  quadruple-nested  computational  grids.  All  four  domains 
have  a  mesh  size  of  55x55  points,  and  grid  increments  of  67.5, 22.5, 7.5,  and  2.5  km, 
respectively,  for  the  domains  1,2, 3,  and  4.  Forecast  computations  were  carried  out  in 
parallel  modes  using  16  processors.  The  U.  S.  Army  HPC  computer  was  used  for  this 
study.  For  this  evaluation  study,  the  forecast  data  for  domains  3  and  4  were  compared 
with  observed  data. 

BFM  calculations  were  made  independently  over  two  model  domains  with  grid 
increments  of  5  and  2.5  km,  both  centered  at  Salt  Lake  City,  UT.  Surface  meteorological 
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data  forecasted  by  the  models  were  statistically  compared  with  Mesowest  network  data. 

The  major  findings  of  this  study  included: 

1 .  There  are  no  significant  statistical  differences  in  forecast  skills  between  MM5 
domain  3  (7.5  km  grid  increment)  and  domain  4  (2.5  km  grid  increment). 
Similarly,  the  forecast  skills  of  two  BFM  (5  and  2.5  km)  grid  increments  are 
statistically  similar.  It  can  be  concluded  that  reducing  the  grid  increment  from  7.5 
to  2.5  km  for  MM5,  and  from  5.0  to  2.5  km  from  BFM  does  not  produce 
significant  improvement  of  forecast  results. 

2.  Both  MM5  and  the  BFM  produced  more  accurate  forecast  results  for  surface 
temperature  than  for  dew  point  temperature  and  wind  vector.  The  AD  values  of 
temperature  (approximately  2  °C )  produced  by  both  models  are  compatible  with 
the  values  reported  by  Mass  et  al.  (2002).  Over  the  complex  terrain  of  Utah,  both 
MM5  and  the  BFM  have  difficulties  in  producing  good  forecast  results  for  wind 
vector.  For  both  models,  the  MWDDF  between  forecast  and  observation  was 
greater  than  60°,  and  absolute  wind  speed  differences  were  around  2  m/sec  for  the 
entire  forecast  hour.  In  a  similar  study  over  western  Washington,  Mass  et  al. 
(2002)  reported  that  the  operational  MM5,  triply-nested  with  36, 12,  and  4  km 
grid  increments,  produced  values  of  approximately  40°  for  MWDDF,  and 
approximately  4  knots  (~  2  m/sec)  for  absolute  difference.  The  terrain 
complexities  of  the  model  domains  over  Utah  may  be  the  reason  for  these 
differences. 

3 .  For  all  of  the  surface  meteorological  parameters  of  temperature,  dew  point 
temperature  and  wind  vector,  MM5  produced  statistically  better  forecast  skills 
than  the  BFM. 

4.  Both  MM5  and  the  BFM  showed  better  forecasting  statistics  during  the  period  of 
April- June  2002  than  during  the  period  of  January-March  2002.  During  the 
warmer  period  of  April-June  2002,  local  heating  and  cooling  effects  might  have 
been  more  significant,  influential  factors  for  surface  meteorological  parameters 
than  during  the  colder  period  of  January-March  2002.  A  similar  study  for  the 
summer  season  is  planned. 
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Appendix  A  Statistical  Diagrams  Displaying  the  Difference  Between 
MM5  Forecast  Data  and  Observation  Data  for  the  Period  of 
January-March  2002. 

This  appendix  is  part  of  ARL-TR-2928,  Evaluation  Study  of  Mesoscale  Models  MM5 
and  BFM  Over  the  Model  Domains  of  Utah  Using  Surface  Meteorological  Data  by 
Mesowest. 


Observa t i on 


Figure  Al.  Difference  between  MM5  forecast  data  and 
observation  data  for  temperature  (domain  3). 
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data  for  wind  vector  y,  component  v  (domain  3). 


Observa t i on 


Figure  A8.  Difference  between  MM5  forecast  data  and  observation 
data  for  wind  vector  y,  component  v  (domain  3). 


Observat i on 


Figure  A9.  Difference  between  MM5  forecast  data  and  observation 
data  for  wind  speed  (domain  3). 
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Figure  A 10.  Difference  between  MM5  forecast  data  and  observation 
data  for  wind  speed  (domain  4). 
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Figure  All.  Time  series  of  statistical  parameters  between  MM5  temperature 
forecast  data  and  surface  observation  data  (domain  3). 
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FigureA12.  Time  series  of  statistical  parameters  between  MM5  temperature 
forecast  data  and  surface  observation  data  (domain  4). 


Figure  A13.  Time  series  of  statistical  parameters  between  MM5  dew  point  temperature  forecast 
data  and  surface  observation  data  (domain  3). 
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Figure  A15.  Time  series  of  statistical  parameters  between  MM5  wind  vector  x,  component  u 
forecast  data  and  surface  observation  data  (domain  3). 
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Figure  A16.  Time  series  of  statistical  parameters  between  MM5  wind  vector  x,  component  u 
forecast  data  and  surface  observation  data  (domain  4). 


Figure  A17.  Time  series  of  statistical  parameters  between  MM5  wind  vector  y,  component  v  forecast 
data  and  surface  observation  data  (domain  3). 
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Figure  A 18.  Time  series  of  statistical  parameters  between  MM5  wind  vector  y,  component  v 
forecast  data  and  surface  observation  data  (domain  4). 
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Figure  A 19.  Time  series  of  statistical  parameters  between  MM5  wind  speed  forecast  data  and  surface 
observation  data  (domain  3). 
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Figure  A20.  Time  series  of  statistical  parameters  between  MM5  wind  speed  forecast  data  and  surface 
observation  data  (domain  4). 
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Figure  A21.  Time  series  of  statistical  parameters  between  MM5  rmsve  and  mean  wind  direction 
forecast  data  and  surface  observation  data  (domain  3). 


41 


Figure  A22.  Time  series  of  statistical  parameters  between  MM5  rmsve  and  mean  wind  direction 
forecast  data  and  surface  observation  data  (domain  4). 
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Appendix  B  Statistical  Diagrams  Displaying  the  Difference  Between 
MM5  Forecast  Data  and  Observation  Data  for  the  Period  of  April- 
June  2002. 


This  appendix  is  part  of  ARL-TR-2928,  Evaluation  Study  of  Mesoscale  Models  MM5 
and  BFM  Over  the  Model  Domains  of  Utah  Using  Surface  Meteorological  Data  by 
Mesowest. 
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Figure  Bl.  Difference  between  MM5  forecast  data  and 
observation  data  for  temperature  (domain  3). 
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Figure  B2.  Difference  between  MM5  forecast  data  and 
observation  data  for  temperature  (domain  4). 


Figure  B3.  Difference  between  MM5  forecast  data  and 
observation  data  for  temperature  (domain  3). 
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Figure  B4.  Difference  between  MM5  forecast  data  and 
observation  data  for  temperature  (domain  4). 


Figure  B5.  Difference  between  MM5  forecast  data  and 

observation  data  for  wind  vector  x,  component  u 
(domain  3). 
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Figure  B6.  Difference  between  MM5  forecast  data  and 

observation  data  for  wind  vector  x,  component  u 
(domain  4). 


Figure  B7.  Difference  between  MM5  forecast  data  and 

observation  data  for  wind  vector  y,  component  v 
(domain  3). 
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Figure  B8.  Difference  between  MM5  forecast  data  and 

observation  data  for  wind  vector  y,  component  v 
(domain  4). 


Figure  B9.  Difference  between  MM5  forecast  data  and 
observation  data  for  wind  speed  (domain  3). 


Observat  f  on 


Figure  BIO.  Difference  between  MM5  forecast  data  and 
observation  data  for  wind  speed  (domain  4). 
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Figure  B1 1.  Time  series  of  statistical  parameters  between  MM5  temperature  forecast  data 
and  surface  observation  data  (domain  3). 
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Figure  B12.  Time  series  of  statistical  parameters  between  MM5  temperature  forecast  data 
and  surface  observation  data  (domain  4). 


Figure  B13.  Time  series  of  statistical  parameters  between  MM5  dew  point  temperature 
forecast  data  and  surface  observation  data  (domain  3). 
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Figure  B14.  Time  series  of  statistical  parameters  between  MM5  dew  point  temperature 
forecast  data  and  surface  observation  data  (domain  4). 


Figure  B15.  Time  series  of  statistical  parameters  between  MM5  wind  vector  x, 
component  u  forecast  data  and  surface  observation  data  (domain  3). 
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Figure  B16.  Time  series  of  statistical  parameters  between  MM5  wind  vector  x, 
component  u  forecast  data  and  surface  observation  data  (domain  4). 


Figure  B17.  Time  series  of  statistical  parameters  between  MM5  wind  vector  y, 
component  v  forecast  data  and  surface  observation  data  (domain  3). 
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Figure  B18.  Time  series  of  statistical  parameters  between  MM5  wind  vector  y, 
component  v  forecast  data  and  surface  observation  data  (domain  4). 


Figure  B19.  Time  series  of  statistical  parameters  between  MM5  wind  speed  forecast  data 
and  surface  observation  data  (domain  3). 
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Figure  B20.  Time  series  of  statistical  parameters  between  MM5  wind  speed  forecast  data 
and  surface  observation  data  (domain  4). 
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Figure  B21.  Time  series  of  statistical  parameters  between  MM5  rmsve  and  mean  wind 
direction  forecast  data  and  surface  observation  data  (domain  3). 
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Appendix  C  Statistical  Diagrams  Displaying  the  Difference  Between 
BFM  Forecast  Data  and  Observation  Data  for  the  Period  of 
January-March  2002. 


This  appendix  is  part  of  ARL-TR-2928,  Evaluation  Study  of  Mesoscale  Models  MM5 
and  BFM  Over  the  Model  Domains  of  Utah  Using  Surface  Meteorological  Data  by 
Mesowest. 
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Figure  Cl.  Difference  between  BFM  forecast  data  and 
observation  data  for  temperature  (5  km  grid 
increment). 
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Figure  C4.  Difference  between  BFM  forecast  data  and 
observation  data  for  dew  point  temperature 
(2.5  km  grid  increment). 
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Figure  C5.  Difference  between  BFM  forecast  data  and 

observation  data  for  wind  vector  x  component, 
u  (5  km  grid  increment). 
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Figure  Cl.  Difference  between  BFM  forecast  data  and 

observation  data  for  wind  vector  y  component,  v  (5 
km  grid  increment). 
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Figure  C8.  Difference  between  BFM  forecast  data  and 

observation  data  for  wind  vector  y  component,  v  (2.5 
km  grid  increment). 
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Figure  C9.  Difference  between  BFM  forecast  data  and 
observation  data  for  wind  speed  (5  km  grid 
increment). 
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Figure  CIO.  Difference  between  BFM  forecast  data  and 

observation  data  for  wind  speed  (2.5  km  grid 
increment). 
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Figure  Cl  1.  Time  series  of  statistical  parameters  between  BFM  temperature  forecast  data  and 
surface  observation  data  (5  km  grid  increment). 
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Figure  C12.  Time  series  of  statistical  parameters  between  BFM  temperature  forecast  data  and 
surface  observation  data  (2.5  grid  increment). 


Figure  Cl  3.  Time  series  of  statistical  parameters  between  BFM  dew  point  temperature 
forecast  data  and  surface  observation  data  (5  grid  increment). 
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Figure  Cl 5.  Time  series  of  statistical  parameters  between  BFM  wind  vector  x  component  u  forecast 
data  and  surface  observation  data  (5  grid  increment). 
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Figure  Cl  8.  Time  series  of  statistical  parameters  between  BFM  wind  vector  y  component  v 
forecast  data  and  surface  observation  data  (2.5  grid  increment). 


Figure  Cl  9.  Time  series  of  statistical  parameters  between  BFM  wind  vector  speed  forecast  data 
and  surface  observation  data  (5  grid  increment). 
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Figure  C20.  Time  series  of  statistical  parameters  between  BFM  wind  vector  speed  forecast  data 
and  surface  observation  data  (2.5  grid  increment). 
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Figure  C21.  Time  series  of  statistical  parameters  between  BFM  rmsve  and  mean  wind  direction 
forecast  data  and  surface  observation  data  (5  grid  increment). 
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Figure  C22.  Time  series  of  statistical  parameters  between  BFM  rmsve  and  mean  wind  direction 
forecast  data  and  surface  observation  data  (2.5  grid  increment). 
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Appendix  D  Statistical  Diagrams  Displaying  the  Difference  Between 
BFM  Forecast  Data  and  Observation  Data  for  the  Period  of  April- 
June  2002. 


This  appendix  is  part  of  ARL-TR-2928,  Evaluation  Study  of  Mesoscale  Models  MM5 
and  BFM  Over  the  Model  Domains  of  Utah  Using  Surface  Meteorological  Data  by 
Mesowest. 
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Figure  Dl.  Difference  between  BFM  forecast  data  and 
observation  data  for  temperature  (5  km  grid 
increment). 
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Figure  D2.  Difference  between  BFM  forecast  data  and 
observation  data  for  temperature  (2.5  km  grid 
increment). 
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Figure  D3.  Difference  between  BFM  forecast  data  and 
observation  data  for  dew  point  temperature 
(5  km  grid  increment). 
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Figure  D4.  Difference  between  BFM  forecast  data  and 
observation  data  for  dew  point  temperature 
(2.5  km  grid  increment). 
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Figure  D5.  Difference  between  BFM  forecast  data 
and  observation  data  for  wind  vector  x 
component  u  (5  km  grid  increment). 
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Figure  D7.  Difference  between  BFM  forecast  data  and 

observation  data  for  wind  vector  y  component  v 
(5  km  grid  increment). 
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Figure  D9.  Difference  between  BFM  forecast  data  and 
observation  data  for  wind  speed  (5  km  grid 
increment). 
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Figure  DIO.  Difference  between  BFM  forecast  data  and 

observation  data  for  wind  speed  (2.5  km  grid 
increment). 
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Figure  Dll.  Time  series  of  statistical  parameters  between  BFM  temperature  forecast  data  and  surface 
observation  data  (5  km  grid  increment). 
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Figure  D12.  Time  series  of  statistical  parameters  between  BFM  temperature  forecast  data  and  surface 
observation  data  (2.5  km  grid  increment). 


Figure  D13.  Time  series  of  statistical  parameters  between  BFM  dew  point  temperature  forecast  data 
and  surface  observation  data  (5  km  grid  increment). 
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Figure  D14.  Time  series  of  statistical  parameters  between  BFM  dewpoint  temperature  forecast  data 
and  surface  observation  data  (2.5  km  grid  increment). 
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Figure  D15.  Time  series  of  statistical  parameters  between  BFM  wind  vector  x  component  u  forecast 
data  and  surface  observation  data  (5  km  grid  increment). 
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Figure  D17.  Time  series  of  statistical  parameters  between  BFM  wind  vector  y  component  v  forecast 
data  and  surface  observation  data  (5  km  grid  increment). 
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Figure  D18.  Time  series  of  statistical  parameters  between  BFM  wind  vector  y  component  v  forecast 
data  and  surface  observation  data  (2.5  km  grid  increment). 


Figure  D19.  Time  series  of  statistical  parameters  between  BFM  wind  speed  forecast  data  and  surface 
observation  data  (5  km  grid  increment). 
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Figure  D20.  Time  series  of  statistical  parameters  between  BFM  wind  speed  forecast  data  and  surface 
observation  data  (2.5  km  grid  increment). 


Figure  D21.  Time  series  of  statistical  parameters  between  BFM  rmsve  and  mean  wind  direction 
forecast  data  and  surface  observation  data  (5  km  grid  increment). 
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Figure  D22.  Time  series  of  statistical  parameters  between  BFM  rmsve  and  mean  wind  direction 
forecast  data  and  surface  observation  data  (2.5  km  grid  increment). 
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Acronyms 


AD 

ARL 

BFM 

CC 

HOTMAC 

HPC 

IMETS 

MD 

MM5 

MPI 

MRF 

MWDDF 

NCAR 

NCEP 

NOGAPS 

NORAPS 

PBL 

RMSE 

RMSVE 

UTC 

WSMR 


absolute  difference 
U.S.  Army  Research  Laboratory 
Battlescale  Forecast  Model 
correlation  coefficient 

Higher  Order  Turbulence  Model  of  Atmospheric  Circulation 

High  Power  Computer 

Integrated  Meteorological  System 

mean  difference 

Mesoscale  Model  version  5 

Message  Passage  Interface 

Medium  Range  Forecast 

mean  wind  direction  difference 

National  Center  for  Atmospheric  Research 

National  Center  for  Environmental  Prediction 

Navy  Operational  Global  Atmospheric  Prediction  System 

Navy  Operational  Regional  Atmospheric  Prediction  System 

Planetary  Boundary  Layer 

root  mean  square  error 

root  mean  square  vector  error 

universal  time  coordinate 

White  Sands  Missile  Range 
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